Image data encoding and decoding

ABSTRACT

An image encoding apparatus comprises a selector configured to select, from a set of candidate prediction operations each defining at least a prediction direction, a prediction operation for prediction of samples of a current region of a current image, the current region comprising an array of two or more rows and two or more columns of samples; an intra-image predictor configured to derive predicted samples of the current region with respect to one or more of a group of reference samples of the same image in dependence upon a prediction direction, defined by the selected prediction operation, between a current sample to be predicted and a reference position amongst the reference samples; in which, for at least some of the candidate prediction operations, the group of reference samples comprises two or more parallel linear arrays of reference samples disposed at different respective separations from the current region; a detector configured to detect whether samples corresponding to any of the two or more parallel linear arrays of reference samples are unavailable for use in prediction of samples of the current region and, if any of the two or more parallel linear arrays of reference samples are unavailable, to inhibit selection, by the selector, of a candidate prediction operation dependent upon the unavailable reference samples.

BACKGROUND Reference to Earlier Applications

This application claims priority from GB1717684.3 filed on 27 Oct. 2017and GB1809024.1 filed on 1 Jun. 2018. The contents of both of thesepriority applications are hereby incorporated by reference.

FIELD

This disclosure relates to image data encoding and decoding.

DESCRIPTION OF RELATED ART

The “background” description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent it is described in thisbackground section, as well as aspects of the description which may nototherwise qualify as prior art at the time of filing, is neitherexpressly or impliedly admitted as prior art against the presentdisclosure.

There are several video data encoding and decoding systems which involvetransforming video data into a frequency domain representation,quantising the frequency domain coefficients and then applying some formof entropy encoding to the quantised coefficients. This can achievecompression of the video data. A corresponding decoding or decompressiontechnique is applied to recover a reconstructed version of the originalvideo data.

Current video codecs (coder-decoders) such as those used in H.264/MPEG-4Advanced Video Coding (AVC) achieve data compression primarily by onlyencoding the differences between successive video frames. These codecsuse a regular array of so-called macroblocks, each of which is used as aregion of comparison with a corresponding macroblock in a previous videoframe, and the image region within the macroblock is then encodedaccording to the degree of motion found between the correspondingcurrent and previous macroblocks in the video sequence, or betweenneighbouring macroblocks within a single frame of the video sequence.

High Efficiency Video Coding (HEVC), also known as H.265 or MPEG-H Part2, is a proposed successor to H.264/MPEG-4 AVC. It is intended for HEVCto improve video quality and double the data compression ratio comparedto H.264, and for it to be scalable from 128×96 to 7680×4320 pixelsresolution, roughly equivalent to bit rates ranging from 128 kbit/s to800 M bit/s.

SUMMARY

The present disclosure addresses or mitigates problems arising from thisprocessing.

Respective aspects and features of the present disclosure are defined inthe appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary, but are notrestrictive, of the present technology.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 schematically illustrates an audio/video (AN) data transmissionand reception system using video data compression and decompression;

FIG. 2 schematically illustrates a video display system using video datadecompression;

FIG. 3 schematically illustrates an audio/video storage system usingvideo data compression and decompression;

FIG. 4 schematically illustrates a video camera using video datacompression;

FIGS. 5 and 6 schematically illustrate storage media;

FIG. 7 provides a schematic overview of a video data compression anddecompression apparatus;

FIG. 8 schematically illustrates a predictor;

FIG. 9 schematically illustrates a partially-encoded image;

FIG. 10 schematically illustrates a set of possible intra-predictiondirections;

FIG. 11 schematically illustrates a set of prediction modes;

FIG. 12 schematically illustrates another set of prediction modes;

FIG. 13 schematically illustrates an intra-prediction process;

FIGS. 14 and 15 schematically illustrate a reference sample projectionprocess;

FIG. 16 schematically illustrates a predictor;

FIGS. 17 and 18 schematically illustrate the use of projected referencesamples;

FIG. 19 schematically illustrates a prediction process;

FIGS. 20 to 22 schematically illustrate example interpolationtechniques;

FIGS. 23 to 26 schematically illustrate respective groups of rows andcolumns of reference samples;

FIGS. 27 to 30 schematically represent respective projected versions ofFIGS. 23 to 26;

FIG. 31 schematically represents an intra mode selector;

FIGS. 32 and 33 are respective schematic flowcharts representing methodsof operation of the intra mode selector of FIG. 31;

FIG. 34 schematically represents an intra mode selector;

FIGS. 35 and 36 are respective schematic flowcharts representing methodsof operation of an intra predictor;

FIG. 37 schematically illustrates a part of the functionality of anintra predictor;

FIG. 38 is a schematic flowchart representing operations of thearrangement of FIG. 37;

FIGS. 39 and 40 are schematic flowcharts illustrating respectivemethods;

FIG. 41 schematically illustrates an array of coding units;

FIG. 42 schematically illustrates a sample predictor;

FIGS. 43 and 44 schematically illustrate reference sample storage;

FIG. 45 schematically represents an intra mode selector;

FIGS. 46 to 49 are schematic diagrams illustrating prediction processes;and

FIGS. 50 and 51 are schematic flowcharts illustrating respectivemethods.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings, FIGS. 1-4 are provided to give schematicillustrations of apparatus or systems making use of the compressionand/or decompression apparatus to be described below in connection withembodiments of the present technology.

All of the data compression and/or decompression apparatus to bedescribed below may be implemented in hardware, in software running on ageneral-purpose data processing apparatus such as a general-purposecomputer, as programmable hardware such as an application specificintegrated circuit (ASIC) or field programmable gate array (FPGA) or ascombinations of these. In cases where the embodiments are implemented bysoftware and/or firmware, it will be appreciated that such softwareand/or firmware, and non-transitory data storage media by which suchsoftware and/or firmware are stored or otherwise provided, areconsidered as embodiments of the present technology.

FIG. 1 schematically illustrates an audio/video data transmission andreception system using video data compression and decompression.

An input audio/video signal 10 is supplied to a video data compressionapparatus 20 which compresses at least the video component of theaudio/video signal 10 for transmission along a transmission route 30such as a cable, an optical fibre, a wireless link or the like. Thecompressed signal is processed by a decompression apparatus 40 toprovide an output audio/video signal 50. For the return path, acompression apparatus 60 compresses an audio/video signal fortransmission along the transmission route 30 to a decompressionapparatus 70.

The compression apparatus 20 and decompression apparatus 70 cantherefore form one node of a transmission link. The decompressionapparatus 40 and decompression apparatus 60 can form another node of thetransmission link. Of course, in instances where the transmission linkis uni-directional, only one of the nodes would require a compressionapparatus and the other node would only require a decompressionapparatus.

FIG. 2 schematically illustrates a video display system using video datadecompression. In particular, a compressed audio/video signal 100 isprocessed by a decompression apparatus 110 to provide a decompressedsignal which can be displayed on a display 120. The decompressionapparatus 110 could be implemented as an integral part of the display120, for example being provided within the same casing as the displaydevice. Alternatively, the decompression apparatus 110 maybe provided as(for example) a so-called set top box (STB), noting that the expression“set-top” does not imply a requirement for the box to be sited in anyparticular orientation or position with respect to the display 120; itis simply a term used in the art to indicate a device which isconnectable to a display as a peripheral device.

FIG. 3 schematically illustrates an audio/video storage system usingvideo data compression and decompression. An input audio/video signal130 is supplied to a compression apparatus 140 which generates acompressed signal for storing by a store device 150 such as a magneticdisk device, an optical disk device, a magnetic tape device, a solidstate storage device such as a semiconductor memory or other storagedevice. For replay, compressed data is read from the storage device 150and passed to a decompression apparatus 160 for decompression to providean output audio/video signal 170.

It will be appreciated that the compressed or encoded signal, and astorage medium such as a machine-readable non-transitory storage medium,storing that signal, are considered as embodiments of the presenttechnology.

FIG. 4 schematically illustrates a video camera using video datacompression. In FIG. 4, an image capture device 180, such as a chargecoupled device (CCD) image sensor and associated control and read-outelectronics, generates a video signal which is passed to a compressionapparatus 190. A microphone (or plural microphones) 200 generates anaudio signal to be passed to the compression apparatus 190. Thecompression apparatus 190 generates a compressed audio/video signal 210to be stored and/or transmitted (shown generically as a schematic stage220).

The techniques to be described below relate primarily to video datacompression and decompression. It will be appreciated that many existingtechniques may be used for audio data compression in conjunction withthe video data compression techniques which will be described, togenerate a compressed audio/video signal. Accordingly, a separatediscussion of audio data compression will not be provided. It will alsobe appreciated that the data rate associated with video data, inparticular broadcast quality video data, is generally very much higherthan the data rate associated with audio data (whether compressed oruncompressed). It will therefore be appreciated that uncompressed audiodata could accompany compressed video data to form a compressedaudio/video signal. It will further be appreciated that although thepresent examples (shown in FIGS. 1-4) relate to audio/video data, thetechniques to be described below can find use in a system which simplydeals with (that is to say, compresses, decompresses, stores, displaysand/or transmits) video data. That is to say, the embodiments can applyto video data compression without necessarily having any associatedaudio data handling at all.

FIG. 4 therefore provides an example of a video capture apparatuscomprising an image sensor and an encoding apparatus of the type to bediscussed below. FIG. 2 therefore provides an example of a decodingapparatus of the type to be discussed below and a display to which thedecoded images are output.

A combination of FIGS. 2 and 4 may provide a video capture apparatuscomprising an image sensor 180 and encoding apparatus 190, decodingapparatus 110 and a display 120 to which the decoded images are output.

FIGS. 5 and 6 schematically illustrate storage media, which store (forexample) the compressed data generated by the apparatus 20, 60, thecompressed data input to the apparatus 110 or the storage media orstages 150, 220. FIG. 5 schematically illustrates a disc storage mediumsuch as a magnetic or optical disc, and FIG. 6 schematically illustratesa solid state storage medium such as a flash memory. Note that FIGS. 5and 6 can also provide examples of non-transitory machine-readablestorage media which store computer software which, when executed by acomputer, causes the computer to carry out one or more of the methods tobe discussed below.

Therefore, the above arrangements provide examples of video storage,capture, transmission or reception apparatuses embodying any of thepresent techniques.

FIG. 7 provides a schematic overview of a video data compression anddecompression apparatus.

A controller 343 controls the overall operation of the apparatus and, inparticular when referring to a compression mode, controls a trialencoding processes by acting as a selector to select various modes ofoperation such as block sizes and shapes, and whether the video data isto be encoded losslessly or otherwise. The controller is considered topart of the image encoder or image decoder (as the case may be).Successive images of an input video signal 300 are supplied to an adder310 and to an image predictor 320. The image predictor 320 will bedescribed below in more detail with reference to FIG. 8. The imageencoder or decoder (as the case may be) plus the intra-image predictorof FIG. 8 may use features from the apparatus of FIG. 7. This does notmean that the image encoder or decoder necessarily requires everyfeature of FIG. 7 however.

The adder 310 in fact performs a subtraction (negative addition)operation, in that it receives the input video signal 300 on a “+” inputand the output of the image predictor 320 on a “−” input, so that thepredicted image is subtracted from the input image. The result is togenerate a so-called residual image signal 330 representing thedifference between the actual and projected images.

One reason why a residual image signal is generated is as follows. Thedata coding techniques to be described, that is to say the techniqueswhich will be applied to the residual image signal, tend to work moreefficiently when there is less “energy” in the image to be encoded.Here, the term “efficiently” refers to the generation of a small amountof encoded data; for a particular image quality level, it is desirable(and considered “efficient”) to generate as little data as ispracticably possible. The reference to “energy” in the residual imagerelates to the amount of information contained in the residual image. Ifthe predicted image were to be identical to the real image, thedifference between the two (that is to say, the residual image) wouldcontain zero information (zero energy) and would be very easy to encodeinto a small amount of encoded data. In general, if the predictionprocess can be made to work reasonably well such that the predictedimage content is similar to the image content to be encoded, theexpectation is that the residual image data will contain lessinformation (less energy) than the input image and so will be easier toencode into a small amount of encoded data.

The remainder of the apparatus acting as an encoder (to encode theresidual or difference image) will now be described. The residual imagedata 330 is supplied to a transform unit or circuitry 340 whichgenerates a discrete cosine transform (DCT) representation of blocks orregions of the residual image data. The DCT technique itself is wellknown and will not be described in detail here. Note also that the useof DCT is only illustrative of one example arrangement. Other transformswhich might be used include, for example, the discrete sine transform(DST). A transform could also comprise a sequence or cascade ofindividual transforms, such as an arrangement in which one transform isfollowed (whether directly or not) by another transform. The choice oftransform may be determined explicitly and/or be dependent upon sideinformation used to configure the encoder and decoder.

The output of the transform unit 340, which is to say, a set of DCTcoefficients for each transformed block of image data, is supplied to aquantiser 350. Various quantisation techniques are known in the field ofvideo data compression, ranging from a simple multiplication by aquantisation scaling factor through to the application of complicatedlookup tables under the control of a quantisation parameter. The generalaim is twofold. Firstly, the quantisation process reduces the number ofpossible values of the transformed data. Secondly, the quantisationprocess can increase the likelihood that values of the transformed dataare zero. Both of these can make the entropy encoding process, to bedescribed below, work more efficiently in generating small amounts ofcompressed video data.

A data scanning process is applied by a scan unit 360. The purpose ofthe scanning process is to reorder the quantised transformed data so asto gather as many as possible of the non-zero quantised transformedcoefficients together, and of course therefore to gather as many aspossible of the zero-valued coefficients together. These features canallow so-called run-length coding or similar techniques to be appliedefficiently. So, the scanning process involves selecting coefficientsfrom the quantised transformed data, and in particular from a block ofcoefficients corresponding to a block of image data which has beentransformed and quantised, according to a “scanning order” so that (a)all of the coefficients are selected once as part of the scan, and (b)the scan tends to provide the desired reordering. One example scanningorder which can tend to give useful results is a so-called up-rightdiagonal scanning order.

The scanned coefficients are then passed to an entropy encoder (EE) 370.Again, various types of entropy encoding may be used. Two examples arevariants of the so-called CABAC (Context Adaptive Binary ArithmeticCoding) system and variants of the so-called CAVLC (Context AdaptiveVariable-Length Coding) system. In general terms, CABAC is considered toprovide a better efficiency, and in some studies has been shown toprovide a 10-20% reduction in the quantity of encoded output data for acomparable image quality compared to CAVLC. However, CAVLC is consideredto represent a much lower level of complexity (in terms of itsimplementation) than CABAC. Note that the scanning process and theentropy encoding process are shown as separate processes, but in factcan be combined or treated together. That is to say, the reading of datainto the entropy encoder can take place in the scan order. Correspondingconsiderations apply to the respective inverse processes to be describedbelow.

The output of the entropy encoder 370, along with additional data(mentioned above and/or discussed below), for example defining themanner in which the predictor 320 generated the predicted image,provides a compressed output video signal 380.

However, a return path is also provided because the operation of thepredictor 320 itself depends upon a decompressed version of thecompressed output data.

The reason for this feature is as follows. At the appropriate stage inthe decompression process (to be described below) a decompressed versionof the residual data is generated. This decompressed residual data hasto be added to a predicted image to generate an output image (becausethe original residual data was the difference between the input imageand a predicted image). In order that this process is comparable, asbetween the compression side and the decompression side, the predictedimages generated by the predictor 320 should be the same during thecompression process and during the decompression process. Of course, atdecompression, the apparatus does not have access to the original inputimages, but only to the decompressed images. Therefore, at compression,the predictor 320 bases its prediction (at least, for inter-imageencoding) on decompressed versions of the compressed images.

The entropy encoding process carried out by the entropy encoder 370 isconsidered (in at least some examples) to be “lossless”, which is to saythat it can be reversed to arrive at exactly the same data which wasfirst supplied to the entropy encoder 370. So, in such examples thereturn path can be implemented before the entropy encoding stage.Indeed, the scanning process carried out by the scan unit 360 is alsoconsidered lossless, but in the present embodiment the return path 390is from the output of the quantiser 350 to the input of a complimentaryinverse quantiser 420. In instances where loss or potential loss isintroduced by a stage, that stage may be included in the feedback loopformed by the return path. For example, the entropy encoding stage canat least in principle be made lossy, for example by techniques in whichbits are encoded within parity information. In such an instance, theentropy encoding and decoding should form part of the feedback loop.

In general terms, an entropy decoder 410, the reverse scan unit 400, aninverse quantiser 420 and an inverse transform unit or circuitry 430provide the respective inverse functions of the entropy encoder 370, thescan unit 360, the quantiser 350 and the transform unit 340. For now,the discussion will continue through the compression process; theprocess to decompress an input compressed video signal will be discussedseparately below.

In the compression process, the scanned coefficients are passed by thereturn path 390 from the quantiser 350 to the inverse quantiser 420which carries out the inverse operation of the scan unit 360. An inversequantisation and inverse transformation process are carried out by theunits 420, 430 to generate a compressed-decompressed residual imagesignal 440.

The image signal 440 is added, at an adder 450, to the output of thepredictor 320 to generate a reconstructed output image 460. This formsone input to the image predictor 320, as will be described below.

Turning now to the process applied to decompress a received compressedvideo signal 470, the signal is supplied to the entropy decoder 410 andfrom there to the chain of the reverse scan unit 400, the inversequantiser 420 and the inverse transform unit 430 before being added tothe output of the image predictor 320 by the adder 450. So, at thedecoder side, the decoder reconstructs a version of the residual imageand then applies this (by the adder 450) to the predicted version of theimage (on a block by block basis) so as to decode each block. Instraightforward terms, the output 460 of the adder 450 forms the outputdecompressed video signal 480. In practice, further filtering mayoptionally be applied (for example, by a filter 560 shown in FIG. 8 butomitted from FIG. 7 for clarity of the higher level diagram of FIG. 7)before the signal is output.

The apparatus of FIGS. 7 and 8 can act as a compression (encoding)apparatus or a decompression (decoding) apparatus. The functions of thetwo types of apparatus substantially overlap. The scan unit 360 andentropy encoder 370 are not used in a decompression mode, and theoperation of the predictor 320 (which will be described in detail below)and other units follow mode and parameter information contained in thereceived compressed bit-stream rather than generating such informationthemselves.

FIG. 8 schematically illustrates the generation of predicted images, andin particular the operation of the image predictor 320.

There are two basic modes of prediction carried out by the imagepredictor 320: so-called intra-image prediction and so-calledinter-image, or motion-compensated (MC), prediction. At the encoderside, each involves detecting a prediction direction in respect of acurrent block to be predicted, and generating a predicted block ofsamples according to other samples (in the same (intra) or another(inter) image). By virtue of the units 310 or 450, the differencebetween the predicted block and the actual block is encoded or appliedso as to encode or decode the block respectively.

(At the decoder, or at the reverse decoding side of the encoder, thedetection of a prediction direction may be in response to dataassociated with the encoded data by the encoder, indicating whichdirection was used at the encoder. Or the detection may be in responseto the same factors as those on which the decision was made at theencoder).

Intra-image prediction bases a prediction of the content of a block orregion of the image on data from within the same image. This correspondsto so-called I-frame encoding in other video compression techniques. Incontrast to I-frame encoding, however, which involves encoding the wholeimage by intra-encoding, in the present embodiments the choice betweenintra- and inter-encoding can be made on a block-by-block basis, thoughin other embodiments the choice is still made on an image-by-imagebasis.

Motion-compensated prediction is an example of inter-image predictionand makes use of motion information which attempts to define the source,in another adjacent or nearby image, of image detail to be encoded inthe current image. Accordingly, in an ideal example, the contents of ablock of image data in the predicted image can be encoded very simply asa reference (a motion vector) pointing to a corresponding block at thesame or a slightly different position in an adjacent image.

A technique known as “block copy” prediction is in some respects ahybrid of the two, as it uses a vector to indicate a block of samples ata position displaced from the currently predicted block within the sameimage, which should be copied to form the currently predicted block.

Returning to FIG. 8, two image prediction arrangements (corresponding tointra- and inter-image prediction) are shown, the results of which areselected by a multiplexer 500 under the control of a mode signal 510(for example, from the controller 343) so as to provide blocks of thepredicted image for supply to the adders 310 and 450. The choice is madein dependence upon which selection gives the lowest “energy” (which, asdiscussed above, may be considered as information content requiringencoding), and the choice is signalled to the decoder within the encodedoutput data-stream. Image energy, in this context, can be detected, forexample, by carrying out a trial subtraction of an area of the twoversions of the predicted image from the input image, squaring eachpixel value of the difference image, summing the squared values, andidentifying which of the two versions gives rise to the lower meansquared value of the difference image relating to that image area. Inother examples, a trial encoding can be carried out for each selectionor potential selection, with a choice then being made according to thecost of each potential selection in terms of one or both of the numberof bits required for encoding and distortion to the picture.

The actual prediction, in the intra-encoding system, is made on thebasis of image blocks received as part of the signal 460, which is tosay, the prediction is based upon encoded-decoded image blocks in orderthat exactly the same prediction can be made at a decompressionapparatus. However, data can be derived from the input video signal 300by an intra-mode selector 520 to control the operation of theintra-image predictor 530.

For inter-image prediction, a motion compensated (MC) predictor 540 usesmotion information such as motion vectors derived by a motion estimator550 from the input video signal 300. Those motion vectors are applied toa processed version of the reconstructed image 460 by the motioncompensated predictor 540 to generate blocks of the inter-imageprediction.

Accordingly, the units 530 and 540 (operating with the estimator 550)each act as detectors to detect a prediction direction in respect of acurrent block to be predicted, and as a generator to generate apredicted block of samples (forming part of the prediction passed to theunits 310 and 450) according to other samples defined by the predictiondirection.

The processing applied to the signal 460 will now be described. Firstly,the signal is optionally filtered by a filter unit 560, which will bedescribed in greater detail below. This involves applying a “deblocking”filter to remove or at least tend to reduce the effects of theblock-based processing carried out by the transform unit 340 andsubsequent operations. A sample adaptive offsetting (SAO) filter mayalso be used. Also, an adaptive loop filter is optionally applied usingcoefficients derived by processing the reconstructed signal 460 and theinput video signal 300. The adaptive loop filter is a type of filterwhich, using known techniques, applies adaptive filter coefficients tothe data to be filtered. That is to say, the filter coefficients canvary in dependence upon various factors. Data defining which filtercoefficients to use is included as part of the encoded outputdata-stream.

The filtered output from the filter unit 560 in fact forms the outputvideo signal 480 when the apparatus is operating as a decompressionapparatus. It is also buffered in one or more image or frame stores 570;the storage of successive images is a requirement of motion compensatedprediction processing, and in particular the generation of motionvectors. To save on storage requirements, the stored images in the imagestores 570 may be held in a compressed form and then decompressed foruse in generating motion vectors. For this particular purpose, any knowncompression/decompression system may be used. The stored images arepassed to an interpolation filter 580 which generates a higherresolution version of the stored images; in this example, intermediatesamples (sub-samples) are generated such that the resolution of theinterpolated image is output by the interpolation filter 580 is 4 times(in each dimension) that of the images stored in the image stores 570for the luminance channel of 4:2:0 and 8 times (in each dimension) thatof the images stored in the image stores 570 for the chrominancechannels of 4:2:0. The interpolated images are passed as an input to themotion estimator 550 and also to the motion compensated predictor 540.

The way in which an image is partitioned for compression processing willnow be described. At a basic level, an image to be compressed isconsidered as an array of blocks or regions of samples. The splitting ofan image into such blocks or regions can be carried out by a decisiontree, such as that described in Bross et al: “High Efficiency VideoCoding (HEVC) text specification draft 6”, JCTVC-H1003_d0 (November2011), the contents of which are incorporated herein by reference. Insome examples, the resulting blocks or regions have sizes and, in somecases, shapes which, by virtue of the decision tree, can generallyfollow the disposition of image features within the image. This initself can allow for an improved encoding efficiency because samplesrepresenting or following similar image features would tend to begrouped together by such an arrangement. In some examples, square blocksor regions of different sizes (such as 4×4 samples up to, say, 64×64 orlarger blocks) are unavailable for selection. In other examplearrangements, blocks or regions of different shapes such as rectangularblocks (for example, vertically or horizontally oriented) can be used.Other non-square and non-rectangular blocks are envisaged. The result ofthe division of the image into such blocks or regions is (in at leastthe present examples) that each sample of an image is allocated to one,and only one, such block or region.

The intra-prediction process will now be discussed. In general terms,intra-prediction involves generating a prediction of a current block ofsamples from previously-encoded and decoded samples in the same image.

FIG. 9 schematically illustrates a partially encoded image 800. Here,the image is being encoded from top-left to bottom-right on a block byblock basis. An example block encoded partway through the handling ofthe whole image is shown as a block 810. A shaded region 820 above andto the left of the block 810 has already been encoded. The intra-imageprediction of the contents of the block 810 can make use of any of theshaded area 820 but cannot make use of the unshaded area below that.

In some examples, the image is encoded on a block by block basis suchthat larger blocks (referred to as coding units or CUs) are encoded inan order such as the order discussed with reference to FIG. 9. Withineach CU, there is the potential (depending on the block splittingprocess that has taken place) for the CU to be handled as a set of twoor more smaller blocks or transform units (TUs). This can give ahierarchical order of encoding so that the image is encoded on a CU byCU basis, and each CU is potentially encoded on a TU by TU basis. Notehowever that for an individual TU within the current coding tree unit(the largest node in the tree structure of block division), thehierarchical order of encoding (CU by CU then TU by TU) discussed abovemeans that there may be previously encoded samples in the current CU andavailable to the coding of that TU which are, for example, above-rightor below-left of that TU.

The block 810 represents a CU; as discussed above, for the purposes ofintra-image prediction processing, this may be subdivided into a set ofsmaller units. An example of a current TU 830 is shown within the CU810. More generally, the picture is split into regions or groups ofsamples to allow efficient coding of signalling information andtransformed data. The signalling of the information may require adifferent tree structure of sub-divisions to that of the transform, andindeed that of the prediction information or the prediction itself. Forthis reason, the coding units may have a different tree structure tothat of the transform blocks or regions, the prediction blocks orregions and the prediction information. In some examples such as HEVCthe structure can be a so-called quad tree of coding units, whose leafnodes contain one or more prediction units and one or more transformunits; the transform units can contain multiple transform blockscorresponding to luma and chroma representations of the picture, andprediction could be considered to be applicable at the transform blocklevel. In examples, the parameters applied to a particular group ofsamples can be considered to be predominantly defined at a block level,which is potentially not of the same granularity as the transformstructure.

The intra-image prediction takes into account samples coded prior to thecurrent TU being considered, such as those above and/or to the left ofthe current TU. Source samples, from which the required samples arepredicted, may be located at different positions or directions relativeto the current TU. To decide which direction is appropriate for acurrent prediction unit, the mode selector 520 of an example encoder maytest all combinations of available TU structures for each candidatedirection and select the prediction direction and TU structure with thebest compression efficiency.

The picture may also be encoded on a “slice” basis. In one example, aslice is a horizontally adjacent group of CUs. But in more generalterms, the entire residual image could form a slice, or a slice could bea single CU, or a slice could be a row of CUs, and so on. Slices cangive some resilience to errors as they are encoded as independent units.The encoder and decoder states are completely reset at a slice boundary.For example, intra-prediction is not carried out across sliceboundaries; slice boundaries are treated as image boundaries for thispurpose.

FIG. 10 schematically illustrates a set of possible (candidate)prediction directions. The full set of candidate directions is availableto a prediction unit. The directions are determined by horizontal andvertical displacement relative to a current block position, but areencoded as prediction “modes”, a set of which is shown in FIG. 11. Notethat the so-called DC mode represents a simple arithmetic mean of thesurrounding upper and left-hand samples. Note also that the set ofdirections shown in FIG. 10 is just one example; in other examples, aset of (for example) 65 angular modes plus DC and planar (a full set of67 modes) as shown schematically in FIG. 12 makes up the full set. Othernumbers of modes could be used.

In general terms, after detecting a prediction direction, the systemsare operable to generate a predicted block of samples according to othersamples defined by the prediction direction. In examples, the imageencoder is configured to encode data identifying the predictiondirection selected for each sample or region of the image (and the imagedecoder is configured to detect such data).

FIG. 13 schematically illustrates an intra-prediction process in which asample 900 of a block or region 910 of samples is derived from otherreference samples 920 of the same image according to a direction 930defined by the intra-prediction mode associated with that sample. Thereference samples 920 in this example come from blocks above and to theleft of the block 910 in question and the predicted value of the sample900 is obtained by tracking along the direction 930 to the referencesamples 920. The direction 930 might point to a single individualreference sample but in a more general case an interpolated valuebetween surrounding reference samples is used as the prediction value.Note that the block 910 could be square as shown in FIG. 13 or could beanother shape such as rectangular.

FIGS. 14 and 15 schematically illustrate a previously proposed referencesample projection process.

In FIGS. 14 and 15, a block or region 1400 of samples to be predicted issurrounded by linear arrays of reference samples from which the intraprediction of the predicted samples takes place. The reference samples1410 are shown as shaded blocks in FIGS. 14 and 15, and the samples tobe predicted are shown as unshaded blocks. Note that an 8×8 block orregion of samples to be predicted is used in this example, but thetechniques are applicable to variable block sizes and indeed blockshapes.

As mentioned, the reference samples comprise at least two linear arraysin respective orientations with respect to the current image region ofsamples to be predicted. For example, the linear arrays may be an arrayor row 1420 of samples above the block of samples to be predicted and anarray or column 1430 of samples to the left of the block of samples tobe predicted.

As discussed above with reference to FIG. 13, the reference samplearrays can extend beyond the extent of the block to be predicted, inorder to provide for prediction modes or directions within the rangeindicated in FIGS. 10-12. Where necessary, if previously decoded samplesare not available for use as reference samples at particular referencesample positions, other reference samples can be re-used at thosemissing positions. Reference sample filtering processes can be used onthe reference samples.

A sample projection process is used to project at least some of thereference samples to different respective positions with respect to thecurrent image region, in the manner shown in FIGS. 14 and 15. In otherwords, in embodiments, the projection process and circuitry operates torepresent at least some of the reference samples at different spatialpositions relative to the current image region, for example as shown inFIGS. 14 and 15. Thus at least some reference samples may be moved (forthe purposes at least of defining an array of reference samples fromwhich samples are predicted) with respect to their relative positions tothe current image region. In particular, FIG. 14 relates to a projectionprocess performed for modes which are generally to the left of thediagonal mode (18 in FIG. 11) mainly modes 2 . . . 17, and FIG. 15schematically illustrates a reference sample projection carried formodes 19 . . . 34, namely those generally above the block to bepredicted (to the right of the diagonal mode 18). The diagonal mode 18can be assigned to either of these two groups as an arbitrary selection,such as to the group of modes to the right of the diagonal. So, in FIG.14, when the current prediction mode is between modes 2 and 17 (or theirequivalent in a system such as that of FIG. 12 having a different numberof possible prediction modes), the sample array 1420 above the currentblock is projected to form additional reference samples 1440 in the lefthand column. Prediction then takes place with respect to the linearprojected array 1450 formed of the original left hand column 1430 andthe projected samples 1440. In FIG. 15, for modes between 18 and 34 ofFIG. 11 (or their equivalent in other sets of prediction modes such asthose shown in FIG. 12), the reference samples 1500 in the left handcolumn are projected so as to extend to the left of the referencesamples 1510 above the current block. This forms a projected array 1520.

One reason why projection of this nature is carried out is to reduce thecomplexity of the intra prediction process, in that all of the samplesto be predicted are then referencing a single linear array of referencesamples, rather than referencing two orthogonal linear arrays.

FIG. 16 schematically illustrates a previously proposed predictioncircuitry 600 arranged to carry out the projection process of FIGS. 14and 15, specifically by providing projection circuitry 1610 configuredto carry out a projection process on the reference samples currentlyselected for a block of region to be predicted. The projected referencesamples are stored in a buffer 1620 to be accessed by an intra predictor1630 to generate predicted samples from the projected reference samples.The projection process is carried out according to the prediction modeassociated with the current block to be predicted, using the techniquesdiscussed in connection with FIGS. 14 and 15.

In embodiments, the same projection process is carried out in thedecoder and in the encoder, so that the predicted samples are the samein each instance. Possible variations in operation between the use ofprediction modes which will be referred to as “straight modes” andprediction modes which will be referred to as “curved modes” will now bediscussed.

As further background, FIGS. 17 and 18 schematically illustrate anexample technique by which samples 1900 of a current region 1910 orblock to be predicted, are predicted from reference samples 1920. Inthis example, the reference samples have been projected into a lineararray using the techniques described with reference to FIGS. 14-16above.

A system of (x, y) coordinates is used for convenience, to allowindividual reference or predicted sample positions to be identified. Inthe example of FIG. 17, x coordinates are shown by a row 1930 ofnumbers, and y coordinates are shown by a column 1940 of numbers. So,each reference or predicted sample position has an associated (x, y)designation using the coordinate system.

In the example of FIG. 17, a generally vertical mode (for example, amode which is more vertical than horizontal) 1950, such as mode 23 inthe designation of FIG. 11, noting that a different mode number could beused if the set of modes shown in FIG. 12 were employed, has beenselected for prediction of samples 1900 of the block or region 1910. Asdiscussed above with reference to FIGS. 14-16, such a generally verticalprediction mode is handled by the circuitry of FIG. 16 by projecting theleft column of reference samples into an extension 1960 of the referencesamples above the block or region 1910.

Each of the samples to be predicted 1900 is predicted as follows. Foreach sample to be predicted, there is an associated (x, y) location suchas a location (0, 5) for a sample 1970 or a location (0, 4) for a sample1972. These two samples are used purely by way of example and the sametechnique applies to each of the samples 1900 to be predicted.

The sample positions of the samples 1970, 1972 to be predicted aremapped according to the direction 1950 associated with the currentprediction mode to respective locations or reference positions 1974,1976 among the reference samples. This mapping may be carried out usingan expression such as that shown below, noting that this is a linearexpression with respect to the coordinate system (x, y):

For horizontal modes 2-17 in the notation of FIG. 11:

predicted value(x. y)={1−f(p)}×ref[y+i(p)]+f(p)×ref[y+i(p)+1] withp=A×(x+1)

For vertical modes 18-34 in the notation of FIG. 11:

predicted value(x. y)={1−f(p)}×ref[x+i(p)]+f(p)×ref[x+i(p)+1] withp=A×(y+1)

and where i(p)=floor(p), is the value p rounded down (towards negativeinfinity) to the nearest integer, f(p)=p−i(p) represents the fractionalpart of the value p.

A is an angle parameter indicating the angle of the current mode. Toillustrate, for example, for a horizontal or vertical line, A would be0; for a 45° diagonal line, A would be ±1.

Those skilled in the art would appreciate that integer approximationscan be used to simplify the linear equations, for example, representingthe angle parameter A as a fractional fixed-precision number. In HEVC,the angles have an accuracy of 5 fractional bits.

So, for example, each sample to be predicted is associated with acoordinate position within the current region; and the intra-imagepredictor is configured to detect the reference position for a givensample to be predicted as a function of the coordinate position of thegiven sample to be predicted, the function depending upon the selectedprediction mode. In example arrangements, the reference position 1974,1976 is detected to an accuracy or resolution of less than one sample,which is to say with reference to the reference sample locations (−5,−1) . . . (15, −1), a fractional value is used for the x coordinate ofthe reference position within the projected set of reference samples1920. For example, the reference position could be detected to aresolution of 1/32 of a sample separation, so that the x coordinate ofthe reference positions 1974, 1976 is identified to that resolution. They coordinate of the reference position is −1 in each case, but this isin fact irrelevant to the calculations that then take place, whichrelate to interpolation along the x axis of the reference samples 1920.

The prediction of the predicted values 1970, 1972 is an interpolation ofthe value applicable to the detected x coordinate of the referencesample position 1974, 1976, for example as described above in theformulae shown earlier.

A similar arrangement is shown schematically in FIG. 18, except that agenerally horizontal prediction mode, for example a prediction modewhich is more horizontal than vertical, such as (for example) mode 14 ofthe set shown in FIG. 11 (or a corresponding number for a similar modein the set shown in FIG. 12) having a prediction direction 2000 is used.The selection of the particular prediction mode applies to the whole ofthe block or region 2010 of samples 2020 to be predicted and theparticular example here is chosen purely for the purposes of the presentexplanation.

In the case of a generally horizontal mode, as discussed above, theprojection circuitry shown in FIG. 16 projects those reference samplesfrom above the block or region 2010 to form an extension 2030 ofreference samples to the left of the region. Once again, the derivationof two example samples to be predicted, samples 2032, 2034, is shown,such that the sample position 2032, 2034 are mapped using the direction2000 into reference positions 2036, 2038 amongst the set of referencesamples 2040. Once again, a similar (x, y) coordinate system is used andthe reference positions 2036, 2038 are expressed to a 1/32 sampleresolution in the y-direction. The x coordinate of the reference samplepositions is −1 but this is irrelevant to the process which follows. Thesample values of the samples to be predicted are obtained in the mannerdescribed above.

In these arrangements, the intra predictor 530 provides an example of adetector configured to detect the reference position as an arrayposition, with respect to an array of the reference samples, pointed toby the prediction direction applicable to the current sample to bepredicted; and a filter configured to generate the predicted sample byinterpolation of the array of reference samples at the detected arrayposition. The detector may be configured to detect the array position toan accuracy of less than one sample such as 1/32 sample.

The intra mode selector 520 the selector may be configured to perform atleast a partial encoding to select the prediction mode.

FIG. 19 schematically illustrates a prediction process.

In the arrangements of FIGS. 17 and 18, for example, the referencesamples 1920, 2440 comprised a single row and column of samples aroundthe current region or block to be predicted. In FIGS. 17 and 18, thissingle row and single column were projected to form either an elongatesingle row in FIG. 17 or an elongate single column in FIG. 18. But theorigin of the reference sample in both cases was a single row and columnto the left of and above the current region.

Further possibilities will now be discussed in which, in at least someexample circumstances, multiple rows and/or multiple columns ofreference samples are used.

FIG. 19 schematically illustrates a situation relating to an 8×8 block2050 of reference samples 2055. The example used here is of an 8×8block, but it will be appreciated that the present techniques can applyto various sizes and indeed shapes of blocks. So, the present techniquescould apply to other sizes such as 4×4, 16×16, 32×32, 64×64 blocks orthe like, or to non-square blocks such as (purely by way of example)8×16 or the like. So, references to the 8×8 blocks are purely for thepurposes of illustrative discussion.

In FIG. 19, two rows of reference samples are used above the block orregion 2050 and two columns of reference samples are used to the left ofthe block or region 2050. Purely by way of example, a predictiondirection 2060 is assumed to have been selected for the block 2050. Thiscould correspond, for example, to the mode 2 in the notation of FIG. 11or a corresponding mode in the notation of FIG. 12. The interpolation orprediction of a particular example predicted sample 2065 will bediscussed, but similar techniques apply to each of the samples 2055 tobe predicted in the block or region 2050.

Discussing first the reference samples, it will be seen that thereference samples in use in FIG. 19 comprise a row and column 2070spatially nearest to the block 2050, along with a further row or column2075 next-adjacent to the row and column 2070. It can also be seen thatthe row and column 2075 extends further (to reference samples 2080,2085) than the row and column 2070, in order to allow for predictionover the range of angles corresponding to the prediction modes 2 . . .34 in FIG. 11 or the equivalent in FIG. 12 to be used. The referencesamples 2080, 2085 can simply be drawn from previously decoded referencesamples in the normal way. If they are unavailable (because they havenot yet been decoded) then next-adjacent samples 2081, 2086 can berepeated to form the samples 2080, 2085 respectively, or alternativelyan extrapolation process can be used as discussed below.

Turning to the interpolation of the sample 2065, it can be seen thatapplying the direction 2060 defined by the current prediction modepoints to a reference position 2090 in the first row and column 2070 ofreference samples. Extending the prediction direction points to afurther position 2095 in the second row and column 2075. The referencesamples around these two reference positions have been annotated asreference samples a . . . g for clarity of the following explanation. Itis also assumed, by way of example, that a 3-tap interpolation processsuch as the process discussed above is used to derive a predictedsample. Of course, other interpolation techniques could be used and thefollowing discussion would simply be adapted accordingly.

FIGS. 20-22 relate to various possible techniques which can be appliedby the intra predictor 530 for making use of two rows and columns ofreference samples in the form shown in FIG. 19.

Considering first, FIG. 20, the reference position 2090 is taken intoaccount and the three samples in the row and column 2070 (namely thereference samples b, d, f) and the reference samples in the row andcolumn 2075 (namely the reference samples c, e, g), which is to say thereference samples within a range 2120 pointed to by the predictiondirection in use, are combined. So in this example, pairs of referencesamples, one from each of the rows/columns 2070, 2075, are combined inrespective groups and the resulting combined reference samples are thenused in an interpolation process. The selection of the reference samplesto be combined is based upon the reference position 2090 in therow/column 2070 and separately on the reference position 2095 in therow/column 2075. This means that a range 2100 of reference samples inthe row/column 2070 is used, and (according to the prediction directionin use) a different—or at least potentially different—range 2120 ofreference samples (c, e, g) is used in dependence upon the referenceposition 2095 in the row/column 2075. The combination takes placebetween the pairs of reference samples, which is to say that referencesamples c and b are combined to form a reference sample h; referencesamples e and d are combined to form a reference sample i; and referencesamples g and f are combined to form a reference sample j. The referencesample h, i and j are then processed by (in this example) a three-tapinterpolation process to form a predicted sample p.

The combination applied to the pairs of reference samples (c, b), (e,d), (g, f) is shown by an arbitrary symbol “⊖” to indicate that variouspossibilities exist for the nature of this combination. This combinationcould be a simple arithmetic mean. In other examples, it could be aweighted mean, for example so as to apply a greater weight to thereference samples (b, d, fin this example) spatially closer to the block2050 than the reference samples (c, e, g) spatially further away fromthe block 2050. For example, in the situation of FIG. 19 in which tworows and columns of reference samples are used, the weighting could be0.6 for the closer reference samples and 0.4 for the further-away samplein each pair, so that (for example) h=0.4c+0.6b. In a situation such asone to be discussed below in which (for example) three or four rows andcolumns of reference samples are used, a weighting could be applied in asimilar manner as follows (where Rn is a reference sample in row/columnn, where n=1 for the row/column spatially closest to the block or regionto be predicted):

Three rows/columns:

combined reference sample=0.5R1+0.3R2+0.2R3

Four rows/columns:

combined reference sample=0.5R1+0.25R2+0.15R3+0.1R4

Of course, other combinations, or indeed equal combinations, could beused.

So, in the example above, a combination process such as an arithmeticmean or a weighted arithmetic mean is used to combine reference samplesin the rows/columns 2070, 2075 and then the predicted sample generationprocess such as a three-tap interpolation process is used on thecombined values.

As discussed below in connection with FIG. 36, this combination can bedone “in advance” so that a first stage of operation of the intrapredictor 530 can be to combine the multiple rows and columns ofreference samples according to the currently selected predictiondirection, so that that prediction sample generation process proceedswith respect to the combined values as though they were the referencesamples themselves, providing an example in which in which theintra-image predictor is configured to combine the two or more parallellinear arrays of reference samples to form a linear array of referencesamples.

Therefore, FIG. 20 provides an example in which the intra-imagepredictor is configured to combine two or more sets of reference samples(such as (a, c, e) and (b, d, f) in FIG. 20, or (c, e, g) and (b, d, f)in FIG. 20) to derive intermediate reference sample values (h, I, j),and to derive the predicted sample p from the intermediate referencesample values. In example arrangements, the intra-image predictor isconfigured to derive the predicted samples by interpolating amongst theintermediate reference samples. For example, each set of referencesamples may comprises samples from a respective one or the two or moreparallel arrays 2070, 2075 of reference samples. In the case of the useof the samples (c, e, g) in FIG. 20, based around the reference position2095, this is an example in which each set of reference samplescomprises a set, in the respective array of reference samples, pointedto by the prediction direction. In some examples, which the intra-imagepredictor is configured to combine the reference sample values accordingto a weighted combination, in which a weighting applied to a referencesample value decreases with increasing separation of the set ofreference samples containing that reference sample value, from thecurrent region or the current sample to be predicted. For example, theweighting of 0.6 can be used for the reference samples (b, d, f) fromthe array 2070, and the weighting 0.4 can be used for the referencesamples (a, c, e) or (c, e, g) from the array 2075.

In alternative examples, rather than mixing c,b->h a weighted mix(interpolation) of two or more of {a,c,e,g} can be used such that theinterpolated value is spatially aligned with b according to theprediction direction 2060, Then h can be a 50:50 or other weighted mixbetween b and interp(two or more of {a,c,e,g}).

Effectively this involves interpolating the whole column 2075 such thatit is aligned with 2070 according to the direction 2060. Theinterpolated column can then be mixed (by 50:50, 25:75 or anotherweighting) with the column 2070.

During the interpolation process described above, since the projectionof the column 2075 to be spatially aligned with samples of the column2070 according to the current prediction direction will requireinterpolation, supersampling (so as to generate interpolated samples ata smaller spatial resolution than the original reference samples) couldbe used to reduce any negative impact of the interpolation process(since interpolation can in some situations soften data or reduces highfrequency detail).

Another option is to use so-called non-uniform sampling, to combine thetwo columns into a supersampled data set. The phasing of the tworegularly set of sampled values is determined by the angle of thecurrently selected prediction direction. To avoid effects of noise, thenew reference samples may be low-pass filtered, either in a subsequentprocess, or as part of the supersampling process.

In another example method of operation, each row/column 2070, 2075 isused individually to generate an intermediate predicted sample value,and the intermediate predicted sample values are then combined.

Therefore these arrangements provide examples in which each set ofreference samples comprises a set, in the respective array of referencesamples, or of values interpolated from the respective array ofreference samples, pointed to by the prediction direction.

Looking first at FIG. 21, this relates to the use of the range 2100 ineach row/column being aligned only with the reference position 2090 inthe row/column 2070, so that the reference samples a, c and e arecombined (for example, by the three-tap interpolation process) toproduce a first intermediate predicted sample p1. The reference samplesb, d and f in the row/column 2070 are combined by a similar process toproduce a second intermediate predicted value p2. The values p1 and p2can then be combined, for example, by an arithmetic mean or a weightedmean (for example, as discussed above, placing a greater weight such as0.6 on the intermediate predicted sample value p2 and a reduced weightsuch as 0.4 on the intermediate predicted sample value p1, given that itis generated from reference samples further away from the block 2050) togenerate the final predicted sample value p 2200.

A similar arrangement is shown in FIG. 22, but making use of the rangeof reference samples 2100 in the row/column 2070 and the range 2120 inthe row/column 2075, which is to say that reference samples in therow/column 2075 around the reference position 2095 in that row/columnare used.

So, the first intermediate predicted sample value p1 is generated fromthe reference samples c, e and g and the second intermediate predictedsample value p2 is generated from the reference samples b, d and f. Asbefore, these can be combined by any of the processes discussed above toform the final predicted sample value p.

The examples discussed with reference to FIGS. 20-23 relate to a pair ofrows/column 2070, 2075. If more than two rows/columns are use, theneither the processes discussed above could be applied individually. So,in the case of FIG. 20, for n rows/columns, where n is at least two, allof the reference samples within respective ranges 2100, 2120 and thelike of each individual row/column are combined to form a set of threeintermediate reference samples h, i, j which are then combined. In thecase of FIGS. 21 and 22, for n rows/columns, where n is at least two, nintermediate predicted sample values are generated and are thencombined, for example using a weighted combination.

In example arrangements, the controller 343 can vary the weightingaccording to one or more properties or parameters of the currentinterpolation process. For example, such a parameter may be the blocksize of the current block to be interpolated. The weighting could bedetected by the controller 343 from a predetermined or programmable (forexample via parameter sets communicated as part of the compressed datastream) set of weight values, or derived using a predetermined orprogrammable function. An example of such a relationship (whetherdefined by a look-up or a function) is:

block size up to a threshold block size (such as a threshold of 4×4, 8×8or (in the case of non-square blocks) one dimension being up to 8samples): weighting is 25:75 (25% for the further row/column ofreference samples or interpolated samples derived from them and 75% forthe closer row/column); or

block size greater than the threshold block size: weighting is 50:50

In other examples, the (or a) parameter may represent a spatialseparation, in sample rows or columns or along the prediction direction,of the current sample to be interpolated from the nearest row/column ofreference samples. In the example of FIG. 19, the sample position 2065is in the fourth column of samples to be interpolated, starting at thereference column 2070. A mapping can be used between weightings andcolumn separation (or row separation in the case of a generally verticalprediction direction, such as: m=weighting applied to nearer column/rowof reference samples or to interpolated samples derived from them;

n=weighting applied to farther column/row of reference samples, or tointerpolated samples derived from them;

s=separation of current sample position from nearest reference incolumns/rows

bs=block size in that dimension (in columns or rows, whichever is usedto define s) For example:

m:n=s: (s+1)

Or:

m=0.25+(0.25*s/bs); n=(1−m)

The weighting used can be generated by applying two or more of thesefunctions as discussed, for example with m,n being respective productsof a weighting m,n derived by block size and a weighting m,n derived bysample position.

In other words, the influence or contribution of a non-adjacent row orcolumn of reference samples increases as the separation of the sampleposition to be predicted from that row/column increases. For example,for samples to be predicted which are adjacent to the nearest row/columnof reference samples, the influence of another (further away) row/columnof reference samples may be expected to be lower than if the sample tobe predicted is a long way (say, 8 pixels or more) from the row/columnof reference samples adjacent to the current block, then the influenceof the non-adjacent (such as next) row/column of reference samples maybe expected to be larger.

Therefore, in examples, the intra-image predictor is configured tocombine the intermediate sample values according to a weightedcombination, in which a weighting applied to an intermediate samplevalue derived from reference samples non-adjacent to the current imageregion increases with increasing separation of the set of referencesamples, from which that intermediate sample value as generated, fromthe current sample to be predicted.

In examples, the intra-image predictor is configured to combine thereference sample values according to a weighted combination, in which aweighting applied to a reference sample value non-adjacent to thecurrent image region increases with increasing separation of the set ofreference samples containing that reference sample value, from thecurrent sample to be predicted.

Various different options of these combinations can be tested as trialencodings and one selected, for example according to a lowest sum ofabsolute differences (SAD) amongst those tested, for use in encoding,with the selection being indicated by parameter data communicated aspart of the compressed data stream.

Alternatively reference samples in sub-groups in rows/columns could becombined using the techniques of FIG. 20, to form sub-combinations whichcan then be processed using the techniques shown in FIGS. 21 and 22. Anexample of this arrangement is given below for an example arrangement offour rows/columns of reference samples, numbered 1-4, where row/column 1is spatially closest to the current block or region:

Rows/columns 1 & 2:

-   -   generate first combined reference samples as in FIG. 20    -   generate a first intermediate predicted sample value from the        first combined reference sample values

Rows/columns 3 & 4:

-   -   generate second combined reference samples as in FIG. 20    -   generate a second intermediate predicted sample value from the        second combined reference sample values

Then:

-   -   generate a final predicted sample value p from the first and        second intermediate predicted sample values.

Various options will now be discussed relating to the number of rows andcolumns of reference samples. Again, as before, the examples arediscussed with relation to an 8×8 block 2400 of samples to be predicted,but the same techniques are applicable to other sizes and/or shapes ofblocks.

Note however that in some examples, certain block sizes and/or shapesmay be excluded or restricted in their application of the presenttechniques, for example small blocks, such as blocks having eitherdimension equal to four samples or fewer.

Also, in the operation of an intra mode selector as discussed below,certain directional modes may be excluded from the present techniques.

Therefore, FIGS. 21 and 22 provide examples in which the intra-imagepredictor is configured to derive the predicted samples by interpolatingamongst one or more sets of reference samples. For example, theintra-image predictor can be configured to interpolate amongst two ormore sets of reference samples (such as (a, c, e) and (b, d, f) in FIG.21, or (c, e, g) and (b, d, f) in FIG. 22) to derive a respectiveintermediate sample value p1, p2 from each set of reference samples, andto combine the intermediate sample values to derive the predicted samplep. In example arrangements set of reference samples comprises samplesfrom a respective one or the two or more parallel arrays 2070, 2075 ofreference samples. In the example of FIG. 22, based around the referencepositions 2090, 2095, each set of reference samples comprises a set, inthe respective array of reference samples, pointed to by the predictiondirection. As discussed above, the intra-image predictor 530 can beconfigured to combine the intermediate sample values according to aweighted combination, in which a weighting applied to an intermediatesample value decreases with increasing separation of the set ofreference samples, from which that intermediate sample value asgenerated, from the current region (so that in the example given above,a weighting of 0.6 is applied to the closer array 2070 and a weightingof 0.4 is applied to the further array 2075) or the current sample to bepredicted.

FIG. 23 schematically illustrates a single row/column 2410 of referencesamples. If this was the only option available to the intra predictor,the operation would correspond to a previously proposed intra predictor,but the use of a single row/column 2410 could be provided in an intrapredictor forming an embodiment of the present disclosure in the contextof its optional selection (by the intra mode selector) in combinationwith the optional selection (by the intra mode selector) of one or moreother techniques shown in FIGS. 24 to 26.

FIG. 24 shows a pair of rows/columns 2500, 2510 in which, as discussedabove, the row/column 2510 is extended by one or more samples 2520, 2530so as to allow for the use of the full range of prediction directionsdiscussed above.

Similarly, FIG. 25 schematically represents three rows/columns ofreference samples 2600, 2610, 2620. The row/column 2620 is extended byone or more samples 2630, 2640 with respect to the second row/column2610, for the same reasons.

Finally as an example, although not representing a limit on the numberof rows/columns which can be used, FIG. 23 schematically illustratesfour rows/columns of reference samples, mainly rows/columns 2700, 2710,2720, 2730. Once again, the row/column 2730 is extended by one or morereference samples 2740, 2750 with respect to the third row/column 2720,for the same reasons as those discussed above.

FIGS. 27-30 show, for the example cases of FIGS. 23-26 respectively, aprojected version of the multiple rows/columns of reference samples(shown as shaded blocks in FIGS. 27-30). Note that in at least someexamples, the projection process can be dependent upon the predictiondirection in use, so a single example for an arbitrary predictiondirection is employed in FIGS. 27-30. With respect to the projectedreference sample, a similar technique employed to that in FIG. 17 can beused to derive reference positions and reference samples to apply thetechnique of FIG. 19.

FIG. 31 schematically represents at least part of an operation of anintra mode selector such as the intra mode selector 520 of FIG. 8described above.

The intra mode selector can operate to detect an appropriate mode foruse in intra prediction of a current block or region with respect to aset of reference samples 3205. Various techniques have been proposed forachieving this, such as (at least partial) trial encoding and/oranalysis of properties of the reference samples, to select theprediction operation amongst the candidate prediction operations.

In the present examples, any of these techniques can be used, with (insome example arrangements) the techniques being repeated, or applied inmultiple instances, in respect of multiple permutations of the number ofrows/columns of reference samples.

Here, the term “permutation” is used to indicate a group of rows/columnsof reference samples. The group might include a row/column spatiallynearest to the current block or region, and zero or more next-adjacentrows/columns each progressively spatially further away from the currentblock or region.

The term “prediction operation” can be used to describe a mode ordirection and/or an associated permutation of rows/columns. Therefore,the intra-mode selector can, in example embodiments, be configured toselect, from a set of candidate prediction operations each defining atleast a prediction direction, a prediction operation for prediction ofsamples of a current region of a current image, the current regioncomprising an array of two or more rows and two or more columns ofsamples. The intra-image predictor is configured to derive predictedsamples of the current region with respect to one or more of a group ofreference samples of the same image in dependence upon a predictiondirection, defined by the selected prediction operation, between acurrent sample to be predicted and a reference position amongst thereference samples. For at least some of the candidate predictionoperations, the group of reference samples comprises two or moreparallel linear arrays (such as rows, columns, rows and columns (notingthat a row and column are still a linear array even with a “corner”),and/or projected arrays) of reference samples disposed at differentrespective separations from the current region.

The intra mode selector comprises a mode properties detector 3200 whichacts to detect the encoding properties of each mode under test (and, inat least some examples, of each mode with each permutation Pn of numbersn of rows/columns of reference samples available with that mode, where nranges from 1 to a maximum limit of at least two). A coding efficiencydetector 3210 detects the coding efficiency for each mode/permutationtested by the mode properties detector 3200. The coding efficiency canrelate to aspects such as the amount of data needed to encode theresidual image which would arise where that mode and permutation ofrows/columns to be used, along with the amount of data needed to signalthe use of that mode/permutation.

In the selection of a prediction direction, example arrangements testall 34 directional modes with a simple SAD (sum of absolute differences)test to derive a shortlist of modes most likely to be useful for thecurrent block. The shortlist of modes is then tested with a full encodeto select a prediction mode for use.

The current techniques using multiple rows and/or columns of referencesamples may in some situations be slower (or have higher processingrequirements) than using a single row/column. To alleviate this at theencoder side where such a decision is made and then communicated to thedecoder, example arrangements may be arranged to use the multirow/column prediction only for testing the shortlist of modes.

Based on the detection by the coding efficiency detector 3210, a modeselector 3220 selects a prediction mode 3230 to be sent to the intrapredictor 530, and information 3240 is also sent to a mode encoder 3245such as a part of the controller 343 which encodes data 3250 formingpart of the encoded data stream indicating the mode/permutation in use,which is to say, data identifying the prediction operation selected foreach region of the image.

Therefore, this provides an example in which the intra mode selector isconfigured to select amongst two or more groups of reference samples,each group comprising a respective different number of parallel arraysof reference samples.

FIGS. 32 and 33 are respective schematic flow charts representingpossible methods of operation of the intra mode selector of FIG. 31.

In FIG. 32, steps 3300 . . . 3310, carried out in as many instances(whether in series, as schematically illustrated here, or in parallel)as there are available permutations P of rows/columns of referencesamples, relate to the mode properties detector 3200 and codingefficiency detector 3210 detecting a best mode using each possiblepermutation of rows/columns of reference samples. So, a best mode may beselected using a single row/column of reference samples, another bestmode may be selected using two rows/columns of reference samples and soon. Then, at a step 3320, the coding efficiency detector 3210 detectsthe efficiencies of each of the modes and permutations detected at thestep 3300 . . . 3310 so that at a step 3330 the mode selector 3220selects a single mode and permutation of rows/columns and the modeencoder 3245 encodes information defining the selected mode andpermutation.

In an alternative mode of operation in FIG. 33, an initial stage duringthe encoding process is to establish use of a particular permutation ofn rows/columns (where n is at least two, so the permutation might be tworows/columns) and communicate this to the decoder in, for example, aparameter set, at a step 3400. Then, at a step 3410, the mode propertiesdetector 3200, the coding efficiency detector 3210 and the mode selector3220 cooperate to select a best prediction mode using the particularestablished permutation of rows/columns and at a step 3420 the modeencoder 3425 encodes information defining that mode.

At the decoder side, an intra mode selector is shown by way of examplein FIG. 34, comprising an encoded data detector 3500 which detects datasuch as the data 3250 in the encoded data stream defining a particularmode (and optionally a set of rows/columns) to be used, and a modeselector 3510 provides information 3520 to the intra predictor 530indicating which mode and permutation of rows/columns to use, that is tosay, it is configured to detect encoded data identifying the predictionoperation selected for each region of the image.

FIGS. 35 and 36 are respective flow charts schematically representingmethods of operation of an intra predictor such as the intra predictor530 of FIG. 8.

FIG. 35 relates primarily to the operations shown in FIGS. 21 and 22above, in which, at a step 3600, an intermediate predicted sample valuesuch as the intermediate predicted sample value p2 is interpolated froma first set of reference samples, for example the set in a closestrow/column of reference samples to the current block or region, andthen, using as many iterations as there are rows/columns of referencesamples, a step 3610 represents the interpolation of an intermediatepredicted sample value p1 from each remaining set of reference samplesfrom a respective row/column of reference samples. As discussed above,for those rows/columns of reference samples further away from thecurrent block or region than the closest row/column of reference sample,the selection of which reference samples to use could be, for example,by detecting a reference position in each row/column according to theprediction direction or by using corresponding locations to thoseidentified in the closest row/column. Then, at a step 3620, theintermediate predicted sample values are combined, for example by aweighted mean, to generate the final predicted sample value.

FIG. 36 relates more to the process shown in FIG. 20, in which at a step3700, respective members of the two or more sets of reference samples(with sets other than the set in the closest row/column being definedfor example by tracking a further reference position along theprediction direction) into a combined set (h, i, j of FIG. 20), and theninterpolating a predicted sample value p from the combined set a step3710.

As an optional feature of the operation of the intra predictor, FIG. 37schematically illustrates a technique for providing the additionalreference samples 2520, 2530, 2630, 2640, 2740, 2750, or at least someof them, without necessarily having to simply repeat nearby or adjacentreference samples.

Referring to FIG. 37, a missing reference sample detector 3800 detectsthat a reference sample is not available. This could be because a sampleat that position has not yet necessarily been decoded. In some examples,the sample has not been decoded. In other examples, a sample could havebeen decoded, but because it lies outside a current coding unit it couldbe that the hardware requirements such as buffering needed to make thatreference sample available for the current prediction would beunreasonably high and so a design decision is taken such that the sampleis not provided for prediction. For example, referring to FIGS. 25 and26, these considerations may indicate that the samples 2530, 2640, 2642are not available if they are outside the current coding unit. Toaddress these issues, in example arrangements an extrapolator 3810generates the required value 3820 using an extrapolation process basedon multiple ones of the available reference samples. Operation of thisarrangement is shown by schematic flowchart of FIG. 38 in which a step3900 relates to the detection by the detector 3800 of the missingreference samples and a step 3910 relates to the extrapolation by theextrapolator 3810 of the required values.

Further examples illustrating the use of missing reference sampledetection will be discussed below.

FIG. 39 is a schematic flowchart illustrating an image encoding methodcomprising:

selecting (at a step 4000), from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples; and

deriving (at a step 4010) intra-image predicted samples of the currentregion with respect to one or more of a group of reference samples ofthe same image in dependence upon a prediction direction, defined by theselected prediction operation, between a current sample to be predictedand a reference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region.

FIG. 40 is a schematic flowchart illustrating an image encoding methodcomprising:

selecting (at a step 4100), from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples; and

deriving (at a step 4110) intra-image predicted samples of the currentregion with respect to one or more of a group of reference samples ofthe same image in dependence upon a prediction direction, defined by theselected prediction operation, between a current sample to be predictedand a reference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region.

In at least some of the example arrangements discussed above, the use ofmultiple rows or columns of reference samples may be constrained toinstances where the “extra” (non-adjacent) rows or the multiple columns(or both) lie in the same coding tree unit as the current block. Theseconditions can be applied individually or together. Such arrangementswill now be described further.

FIG. 41 schematically illustrates an array of coding units, in anexample arrangement in which the image encoding apparatus is configuredto encode the current image as successive coding units 4100 of samples.

It will be appreciated from the discussion of FIG. 9 above that in atleast some embodiments, coding takes place, coding unit by coding unit,from top left to bottom right of the set of coding units 4100. Thismeans that for image regions at the extreme edges (the top edge and/orthe left edge in this example) of a coding unit, it may be the case thatat least some samples which would be required as reference samples inplural linear arrays as discussed above may or may not be available,depending on how much (if any) of a preceding decoded coding unit hasbeen buffered. For example, for a particular coding unit 4110,prediction at the top 4120 and/or the left 4130 of the coding unit maybe affected in this way. For coding units at the left and/or top of theimage, the required reference sample data may not exist at all. Notethat the top left to bottom right encoding order is merely an example.If another order were used, the edges being considered here may bedifferent to the top and left edges currently under discussion. Notealso that the shape of the coding units shown in the schematic diagramof FIG. 41 may or may not be representative of the actual shape orrelative sizes of the coding units.

FIG. 42 schematically illustrates a sample predictor 530′, similar tothe intra-image predictor 530 of FIG. 8 but explicitly showing a samplestore 4200 to store samples corresponding to a region of the currentimage other than the current region. In particular, the sample store4200 can be arranged to store previously encoded and decoded samplesfrom one or more coding units adjacent to the coding unit in which thecurrent region is located, for use as reference samples in respect of(for example) an image region at a location within the current codingunit adjacent to a previous coding unit.

FIGS. 43 and 44 schematically illustrate reference sample storage by thesample store 4200. In some examples, such as that shown schematically inFIG. 43, the sample store 4200 may be configured as a line store tostore one or more lines or rows of previously encoded and decodedsamples from a coding unit directly above a current coding unit 4310.

Here, it is noted that line stores are considered to be relatively“expensive”, the term implying that they can require a significantamount of memory storage for their implementation, which can in turnrequire a significant amount of logic circuitry in, for example, anintegrated circuit implementation of an encoder and/or decoder. Theprovision of line stores therefore represents a compromise between thepotentially heavy requirements in terms of logic circuitry and thepotential improvements in coding efficiency provided by the presenttechniques in which multiple rows and/or columns of reference samplesmay be used. Such reference samples may be readily available within aparticular coding unit, but access to reference samples from outside thecurrent coding unit can potentially be restricted by the implementation(or lack of implementation) of sample storage such as the sample store4200.

In FIG. 44, a column of samples 4400 from a previously encoded anddecoded coding unit to the left of the current coding unit 4410 isstored for use as reference samples in respect of a left hand column ofregions of the coding unit 4410.

It will be appreciated that the sample store 4200 could store samplesabove (as in FIG. 43) and to the left of (as in FIG. 44) the currentcoding unit. It will be appreciated that one or more rows (and/orcolumns) of samples could be stored. It will be appreciated that asample store 4200 may not in fact be provided.

FIG. 45 schematically represents an intra mode selector arrangement ofthe type shown in FIG. 31, except that the mode selector 3220′ (similarin most respects to the mode selector 3220) is responsive to a detector4500 configured to detect whether samples corresponding to any of thetwo or more parallel linear arrays of reference samples are unavailablefor use in prediction of samples of the current region and, if any ofthe two or more parallel linear arrays of reference samples areunavailable, to inhibit selection, by the intra mode selector 3220′, ofa candidate prediction operation dependent upon the unavailablereference samples.

Note that this can be a separate test and mechanism to the arrangementof FIG. 38. In the present test, it is detected whether any of the twoor more parallel linear arrays of reference samples are unavailable forexample, because the current region is at a peripheral position withrespect to a coding unit and/or image, and/or because any of the two ormore parallel linear arrays of reference samples are not buffered (forexample in line stores). A sample-by-sample extrapolation process of thetype used in FIG. 38 is not applied, but instead, candidate predictionmodes which use the unavailable rows/columns are inhibited from beingselected.

Similarly, the detector 4500 is further configured to detect for acurrent region of a current image which, if any, of the two or moreparallel linear arrays are available for use in prediction of samples ofthe current region and to allow selection as a prediction operation, bythe selector, of a candidate prediction operation dependent upon atleast one of the available parallel linear arrays. Therefore in exampleembodiments the detector 4500 allows or disallows the selection of modesby the selector 3220′ in dependence upon whether, for the current imageregion, the required rows and/or columns of reference samples areavailable (for example, stored or buffered).

The detector 4500 is in turn responsive to configuration data 4510defining the presence or absence of the sample store 4200 and inparticular defining how many (if any) rows and/or columns of samples ofa previous adjacent coding unit it holds for use as reference samples inrespect of image regions of the current coding unit.

Significantly, the configuration data 4510 may define the lower of: (a)which stores are present at the decoder side and (b) which stores areavailable or present at the encoder side. This is because the selectedmode will ultimately need to be implemented at the decoder, but the modecannot be implemented in encoding unless the relevant stores are alsopresent at the encoder side. If the encoded data is to be ultimatelydecoded by a known decoder, the properties of that decoder can beprovided as the configuration data 4510. If however the decoder isunknown, then it may be that the configuration data 4510 defines zerosample stores (even if one or more are present at the encoder) so as notto allow the generation of encoded data that cannot be decoded by anarbitrary decoder. In other examples, if a decoder standard, whether anofficial standard, a proprietary standard or a de facto industrystandard, specifies that (for example) one line store (or a genericnumber n of line stores) must always be present, then for an unknowndecoder the configuration data 4510 can be arranged to specify n linestores, assuming that n line stores are available at the encoder.

In this way, the detector 4500 can be responsive to configuration datadefining one or more parameters of a decoding apparatus to detectwhether samples corresponding to any of the two or more parallel lineararrays of reference samples will be available for use in prediction ofsamples of the current region at the decoding apparatus.

The detector 4500 is also responsive to location information 4520defining the image location of the current image region with respect tothe image and/or the current coding unit. If the current image region isadjacent a left or upper edge of the image then the detector can beconfigured to detect that multiple rows and/or columns of referencesamples are unavailable. If the current image region is adjacent a leftand/or upper edge of a coding unit, then the detector can be configuredto detect that multiple rows and/or columns of reference samples areunavailable unless the configuration data 4510 defines that such samplesare held by sample stores 4200. Therefore, in these examples, thedetector is configured to detect whether, for an image region at alocation within a coding unit adjacent to a previous coding unit, any ofthe two or more parallel linear arrays of reference samples are held bythe sample store.

FIGS. 46 to 49 are schematic diagrams illustrating prediction processes,showing example operations of the detector 4500 to detect whether thecurrent region is adjacent an edge of the current region such thatsample locations corresponding to at least some of the two or moreparallel linear arrays of reference samples lie outside the currentimage.

In FIGS. 46 and 47, an image region 4600 being encoded or decoded is atthe top of its respective coding unit. The boundary with the next-abovecoding unit is schematically represented by a bold line 4610. In thisexample instance a sample store 4200 providing one line store is used,so that one line of samples from the next-above coding unit is availablefor use as reference samples. A second row of reference samples is notavailable above the image region 4600.

In FIG. 46, a single row of reference samples is used for predictionsinvolving a reference position amongst the reference samples above theimage region 4600, such as a reference position 4620. However, because(in the example of FIG. 46) two or more columns of samples 4630 areavailable to the left of the region 4600 for use as reference samples,predictions involving a reference position amongst the reference samplesto the left of the image region 4600, such as a prediction representedby an arrow 4640, use two columns or linear arrays of reference samples.

In FIG. 47, only a single linear array (row, column) is used forprediction for all reference sample locations so that a consistenttechnique is used in respect of all of the predictions. This is doneeven because the second row above is not available; even though thesecond column 4700 is available it is unused.

In FIGS. 48 and 49, once again the current image region 4800 is at thetop of a coding unit (with the boundary 4810 being shown by a boldline). In FIG. 48, one line store is available, and in FIG. 49 two linestores are available.

In FIG. 48, single row prediction is used for reference sample positionsabove the current image region 4800 such as a reference sample position4820, but for reference sample positions to the left, four column 4830prediction (using any of the techniques described above) can d be used.This applies to reference sample locations which do not use row data butonly column data.

In FIG. 49, an extra line store is used (or in other words, two linestores are provided) so a double row/column prediction is used for allsample locations. Although a third and fourth column 4900 are availablethey are unused, so as to provide a consistent approach, although otherexamples could use two rows for reference sample positions above thecurrent region and four columns for reference sample positions to theleft of the current region.

Note that in all of the embodiments described here, for multiplerows/columns the prediction may be a weighted sum but this might be forexample a 0:100:0:0 weighting rather than necessarily a 25:25:25:25weighting (where the four weightings are in order of array separationfrom the current image region, for example, the first, second, third andfourth adjacent rows/columns). This could be implemented as amultiplexer.

Accordingly, in at least FIGS. 46 and 48 the detector 4500 is configuredto detect, separately for reference sample positions disposed verticallyor horizontally with respect to the current image region, whethersamples corresponding to any of the two or more parallel linear arraysof reference samples are unavailable for use in prediction of samples ofthe current region (being, for example, all candidate prediction modesexcept those which rely on line stores or sample provision which are notpresent in respect of the current region) and, if any of the two or moreparallel linear arrays of reference samples are unavailable forreference sample positions disposed vertically or horizontally withrespect to the current image region, to inhibit selection, by theselector, of a candidate prediction operation dependent upon theunavailable reference samples. Note that the detection can take placeregion by region, since the presence or absence of line stores isgenerally relevant only to the top row of regions and/or to modes whichrely on samples above the current region.

In the situation such as at the top of the image (as shown by 4140 inFIG. 41), regardless of what local buffering is present, samples above acurrent region may not be available simply because the current region isat the very top of the image. In such circumstances predetermined value(for example grey) samples can be substituted for the reference samples.The detector 4500 can detect this situation from the locationinformation 4520 and control the detector 4500 so as to disallow use ofmodes other than those which rely on the predetermined referencesamples. Therefore in example embodiments, for at least some of thecandidate prediction operations, the group of reference samplescomprises one or more predetermined values; and the detector 4500 isfurther configured to detect for a current region of a current image ifnone of the two or more parallel linear arrays is available for use inprediction of samples of the current region and to allow selection as aprediction operation, by the selector, of a candidate predictionoperation for that current region dependent upon the predeterminedvalues as reference samples.

At the decoder side, corresponding detections, derivations and controloperations by the detector could be carried out, or alternatively thenature of the encoding could be communicated to the decoder in the sameway as that described with reference to FIG. 34 described above.

FIGS. 50 and 51 are schematic flowcharts illustrating respectivemethods.

FIG. 50 is a schematic flowchart illustrating an image encoding methodcomprising:

selecting (at a step 5000), from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples;

deriving (at a step 5010) intra-image predicted samples of the currentregion with respect to one or more of a group of reference samples ofthe same image in dependence upon a prediction direction, defined by theselected prediction operation, between a current sample to be predictedand a reference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region;

detecting (at a step 5020) whether samples corresponding to any of thetwo or more parallel linear arrays of reference samples are unavailablefor use in prediction of samples of the current region and inhibiting(at a step 5030) selection by the selecting step, if any of the two ormore parallel linear arrays of reference samples are unavailable, of acandidate prediction operation dependent upon the unavailable referencesamples.

FIG. 51 is a schematic flowchart illustrating an image decoding methodcomprising:

selecting (at a step 5100), from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples;

deriving (at a step 5110) intra-image predicted samples of the currentregion with respect to one or more of a group of reference samples ofthe same image in dependence upon a prediction direction, defined by theselected prediction operation, between a current sample to be predictedand a reference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region;

detecting (at a step 5120) whether samples corresponding to any of thetwo or more parallel linear arrays of reference samples are unavailablefor use in prediction of samples of the current region and

inhibiting (at a step 5130) selection by the selecting step, if any ofthe two or more parallel linear arrays of reference samples areunavailable, of a candidate prediction operation dependent upon theunavailable reference samples.

In so far as embodiments of the disclosure have been described as beingimplemented, at least in part, by software-controlled data processingapparatus, it will be appreciated that a non-transitory machine-readablemedium carrying such software, such as an optical disk, a magnetic disk,semiconductor memory or the like, is also considered to represent anembodiment of the present disclosure. Similarly, a data signalcomprising coded data generated according to the methods discussed above(whether or not embodied on a non-transitory machine-readable medium) isalso considered to represent an embodiment of the present disclosure.

It will be apparent that numerous modifications and variations of thepresent disclosure are possible in light of the above teachings. It istherefore to be understood that within the scope of the appendedclauses, the technology may be practised otherwise than as specificallydescribed herein.

Respective aspects and features are defined by the following numberedclauses:

1. An image encoding apparatus comprising:

a selector configured to select, from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples; and

an intra-image predictor configured to derive predicted samples of thecurrent region with respect to one or more of a group of referencesamples of the same image in dependence upon a prediction direction,defined by the selected prediction operation, between a current sampleto be predicted and a reference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region.

2. Apparatus according to clause 1, in which the intra-image predictoris configured to derive the predicted samples by interpolating amongstone or more sets of reference samples.3. Apparatus according to clause 2, in which the intra-image predictoris configured to interpolate amongst two or more sets of referencesamples to derive a respective intermediate sample value from each setof reference samples, and to combine the intermediate sample values toderive the predicted sample.4. Apparatus according to clause 3, in which each set of referencesamples comprises samples from a respective one or the two or moreparallel arrays of reference samples.5. Apparatus according to clause 4, in which each set of referencesamples comprises a set, in the respective array of reference samples,or of values interpolated from the respective array of referencesamples, pointed to by the prediction direction.6. Apparatus according to clause 3 or clause 4, in which the intra-imagepredictor is configured to combine the intermediate sample valuesaccording to a weighted combination, in which a weighting applied to anintermediate sample value decreases with increasing separation of theset of reference samples, from which that intermediate sample value asgenerated, from the current region.7. Apparatus according to clause 3, in which the intra-image predictoris configured to combine the intermediate sample values according to aweighted combination, in which a weighting applied to an intermediatesample value derived from reference samples non-adjacent to the currentimage region increases with increasing separation of the set ofreference samples, from which that intermediate sample value asgenerated, from the current sample to be predicted.8. Apparatus according to any one of the preceding clauses, in which theintra-image predictor is configured to combine two or more sets ofreference samples to derive intermediate reference sample values, and toderive the predicted sample from the intermediate reference samplevalues.9. Apparatus according to clause 8, in which the intra-image predictoris configured to derive the predicted samples by interpolating amongstthe intermediate reference samples.10. Apparatus according to clause 8 or clause 9, in which each set ofreference samples comprises samples from a respective one or the two ormore parallel arrays of reference samples.11. Apparatus according to clause 10, in which each set of referencesamples comprises a set, in the respective array of reference samples,pointed to by the prediction direction.12. Apparatus according to clause 8, in which the intra-image predictoris configured to combine the reference sample values according to aweighted combination, in which a weighting applied to a reference samplevalue decreases with increasing separation of the set of referencesamples containing that reference sample value, from the current region.13. Apparatus according to clause 8, in which the intra-image predictoris configured to combine the reference sample values according to aweighted combination, in which a weighting applied to a reference samplevalue non-adjacent to the current image region increases with increasingseparation of the set of reference samples containing that referencesample value, from the current sample to be predicted.14. Apparatus according to clause 8, in which the intra-image predictoris configured to combine the two or more parallel linear arrays ofreference samples to form a linear array of reference samples.15. Apparatus according to any one of the preceding clauses, in whichthe selector is configured to select amongst two or more groups ofreference samples, each group comprising a respective different numberof parallel arrays of reference samples.16. Apparatus according to any one of the preceding clauses, in whichthe selector is configured to perform at least a partial encoding toselect the prediction operation amongst the candidate predictionoperations.17. Apparatus according to any one of the preceding clauses, in whichthe controller is configured to encode data identifying the predictionoperation selected for each region of the image.18. Video storage, capture, transmission or reception apparatuscomprising apparatus according to any one of the preceding clauses.19. An image decoding apparatus comprising:

a selector configured to select, from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples; and

an intra-image predictor configured to derive predicted samples of thecurrent region with respect to one or more of a group of referencesamples of the same image in dependence upon a prediction direction,defined by the selected prediction operation, between a current sampleto be predicted and a reference position amongst the reference samples;in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region.

20. Apparatus according to clause 19, in which the intra-image predictoris configured to derive the predicted samples by interpolating amongstone or more sets of reference samples.21. Apparatus according to clause 20, in which the intra-image predictoris configured to interpolate amongst two or more sets of referencesamples to derive a respective intermediate sample value from each setof reference samples, and to combine the intermediate sample values toderive the predicted sample.22. Apparatus according to clause 21, in which each set of referencesamples comprises samples from a respective one or the two or moreparallel arrays of reference samples.23. Apparatus according to clause 22, in which each set of referencesamples comprises a set, in the respective array of reference samples,pointed to by the prediction direction.24. Apparatus according to clause 21, in which the intra-image predictoris configured to combine the intermediate sample values according to aweighted combination, in which a weighting applied to an intermediatesample value decreases with increasing separation of the set ofreference samples, from which that intermediate sample value asgenerated, from the current region.25. Apparatus according to clause 19, in which the intra-image predictoris configured to combine two or more sets of reference samples to deriveintermediate reference sample values, and to derive the predicted samplefrom the intermediate reference sample values.26. Apparatus according to clause 25, in which the intra-image predictoris configured to derive the predicted samples by interpolating amongstthe intermediate reference samples.27. Apparatus according to clause 25, in which each set of referencesamples comprises samples from a respective one or the two or moreparallel arrays of reference samples.28. Apparatus according to clause 27, in which each set of referencesamples comprises a set, in the respective array of reference samples,pointed to by the prediction direction.29. Apparatus according to clause 25, in which the intra-image predictoris configured to combine the reference sample values according to aweighted combination, in which a weighting applied to a reference samplevalue decreases with increasing separation of the set of referencesamples containing that reference sample value, from the current region.30. Apparatus according to clause 25, in which the intra-image predictoris configured to combine the two or more parallel linear arrays ofreference samples to form a linear array of reference samples.31. Apparatus according to any one of clauses 19 to 30, in which theselector is configured to select amongst two or more groups of referencesamples, each group comprising a respective different number of parallelarrays of reference samples.32. Apparatus according to any one of clauses 19 to 31, in which thecontroller is configured to detect encoded data identifying theprediction operation selected for each region of the image.33. Video storage, capture, transmission or reception apparatuscomprising apparatus according to any one of clauses 19 to 32.34. An image encoding method comprising:

selecting, from a set of candidate prediction operations each definingat least a prediction direction, a prediction operation for predictionof samples of a current region of a current image, the current regioncomprising an array of two or more rows and two or more columns ofsamples; and

deriving intra-image predicted samples of the current region withrespect to one or more of a group of reference samples of the same imagein dependence upon a prediction direction, defined by the selectedprediction operation, between a current sample to be predicted and areference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region.

35. Computer software which, when executed by a computer, causes thecomputer to carry out a method according to clause 34.36. A machine-readable non-transitory storage medium which storessoftware according to clause 35.37. A data signal comprising coded data generated according to themethod of clause 34.38. An image decoding method comprising:

selecting, from a set of candidate prediction operations each definingat least a prediction direction, a prediction operation for predictionof samples of a current region of a current image, the current regioncomprising an array of two or more rows and two or more columns ofsamples; and

deriving intra-image predicted samples of the current region withrespect to one or more of a group of reference samples of the same imagein dependence upon a prediction direction, defined by the selectedprediction operation, between a current sample to be predicted and areference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region.

39. Computer software which, when executed by a computer, causes thecomputer to carry out a method according to clause 38.40. A machine-readable non-transitory storage medium which storessoftware according to clause 39.41. A video capture apparatus comprising an image sensor and theencoding apparatus of any one of clauses 1-17, decoding apparatus of anyone of clauses 19-32 and a display to which the decoded images areoutput.

Further respective aspects and features are defined by the followingnumbered clauses:

1. An image encoding apparatus comprising:

a selector configured to select, from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples;

an intra-image predictor configured to derive predicted samples of thecurrent region with respect to one or more of a group of referencesamples of the same image in dependence upon a prediction direction,defined by the selected prediction operation, between a current sampleto be predicted and a reference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region; and

a detector configured to detect whether samples corresponding to any ofthe two or more parallel linear arrays of reference samples areunavailable for use in prediction of samples of the current region and,if any of the two or more parallel linear arrays of reference samplesare unavailable, to inhibit selection, by the selector, of a candidateprediction operation dependent upon the unavailable reference samples.

2. Apparatus according to clause 1, in which the detector is furtherconfigured to detect for a current region of a current image which, ifany, of the two or more parallel linear arrays are available for use inprediction of samples of the current region and to allow selection as aprediction operation, by the selector, of a candidate predictionoperation dependent upon at least one of the available parallel lineararrays.3. Apparatus according to clause 1 or clause 2, in which:

for at least some of the candidate prediction operations, the group ofreference samples comprises one or more predetermined values;

the detector is further configured to detect for a current region of acurrent image if none of the two or more parallel linear arrays isavailable for use in prediction of samples of the current region and toallow selection as a prediction operation, by the selector, of acandidate prediction operation for that current region dependent uponthe predetermined values as reference samples.

4. Apparatus according to any one of the preceding clauses, in which thedetector is responsive to an image location of the current region.5. Apparatus according to clause 4, in which the detector is configuredto detect whether the current region is adjacent an edge of the currentregion such that sample locations corresponding to at least some of thetwo or more parallel linear arrays of reference samples lie outside thecurrent image.6. Apparatus according to clause 4 or clause 5, in which:

the image encoding apparatus is configured to encode the current imageas successive coding units of samples;

the image encoding apparatus comprises a sample store to store samplescorresponding to a region of the current image other than the currentregion; and

the detector is configured to detect whether, for an image region at alocation within a coding unit adjacent to a previous coding unit, any ofthe two or more parallel linear arrays of reference samples are held bythe sample store.

7. Apparatus according to any one of the preceding clauses, in which thedetector is configured to detect, separately for reference samplepositions disposed vertically or horizontally with respect to thecurrent image region, whether samples corresponding to any of the two ormore parallel linear arrays of reference samples are unavailable for usein prediction of samples of the current region and, if any of the two ormore parallel linear arrays of reference samples are unavailable forreference sample positions disposed vertically or horizontally withrespect to the current image region, to inhibit selection, by theselector, of a candidate prediction operation dependent upon theunavailable reference samples.8. Apparatus according to any one of the preceding clauses, in which thedetector is responsive to configuration data defining one or moreparameters of a decoding apparatus to detect whether samplescorresponding to any of the two or more parallel linear arrays ofreference samples will be available for use in prediction of samples ofthe current region at the decoding apparatus.9. Video storage, capture, transmission or reception apparatuscomprising apparatus according to any one of the preceding clauses.10. An image decoding apparatus comprising:

a selector configured to select, from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples;

an intra-image predictor configured to derive predicted samples of thecurrent region with respect to one or more of a group of referencesamples of the same image in dependence upon a prediction direction,defined by the selected prediction operation, between a current sampleto be predicted and a reference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region; and

a detector configured to detect whether samples corresponding to any ofthe two or more parallel linear arrays of reference samples areunavailable for use in prediction of samples of the current region and,if any of the two or more parallel linear arrays of reference samplesare unavailable, to inhibit selection, by the selector, of a candidateprediction operation dependent upon the unavailable reference samples.

11. Apparatus according to clause 10, in which the detector is furtherconfigured to detect for a current region of a current image which, ifany, of the two or more parallel linear arrays are unavailable for usein prediction of samples of the current region and to allow selection asa prediction operation, by the selector, of a candidate predictionoperation dependent upon at least one of the available parallel lineararrays.12. Apparatus according to clause 10 or clause 11, in which:

for at least some of the candidate prediction operations, the group ofreference samples comprises one or more predetermined values;

the detector is further configured to detect for a current region of acurrent image if none of the two or more parallel linear arrays isavailable for use in prediction of samples of the current region and toallow selection as a prediction operation, by the selector, of acandidate prediction operation for that current region dependent uponthe predetermined values as reference samples.

13. Apparatus according to any one of clauses 10 to 12, in which thedetector is responsive to an image location of the current region.14. Apparatus according to clause 13, in which the detector isconfigured to detect whether the current region is adjacent an edge ofthe current region such that sample locations corresponding to at leastsome of the two or more parallel linear arrays of reference samples lieoutside the current image.15. Apparatus according to clause 13 or clause 14, in which:

the image decoding apparatus is configured to decode the current imageas successive coding units of samples;

the image decoding apparatus comprises a sample store to store samplescorresponding to a region of the current image other than the currentregion; and

the detector is configured to detect whether, for an image region at alocation within a coding unit adjacent to a previous coding unit, any ofthe two or more parallel linear arrays of reference samples are held bythe sample store.

16. Apparatus according to any one of clauses 10 to 15, in which thedetector is configured to detect, separately for reference samplepositions disposed vertically or horizontally with respect to thecurrent image region, whether samples corresponding to any of the two ormore parallel linear arrays of reference samples are unavailable for usein prediction of samples of the current region and, if any of the two ormore parallel linear arrays of reference samples are unavailable forreference sample positions disposed vertically or horizontally withrespect to the current image region, to inhibit selection, by theselector, of a candidate prediction operation dependent upon theunavailable reference samples.17. Video storage, capture, transmission or reception apparatuscomprising apparatus according to any one of clauses 10 to 16.18. An image encoding method comprising:

selecting, from a set of candidate prediction operations each definingat least a prediction direction, a prediction operation for predictionof samples of a current region of a current image, the current regioncomprising an array of two or more rows and two or more columns ofsamples;

deriving intra-image predicted samples of the current region withrespect to one or more of a group of reference samples of the same imagein dependence upon a prediction direction, defined by the selectedprediction operation, between a current sample to be predicted and areference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region;

detecting whether samples corresponding to any of the two or moreparallel linear arrays of reference samples are unavailable for use inprediction of samples of the current region and

inhibiting selection by the selecting step, if any of the two or moreparallel linear arrays of reference samples are unavailable, of acandidate prediction operation dependent upon the unavailable referencesamples.

19. Computer software which, when executed by a computer, causes thecomputer to carry out a method according to clause 18.20. A machine-readable non-transitory storage medium which storessoftware according to clause 19.21. A data signal comprising coded data generated according to themethod of clause 18.22. An image decoding method comprising:

selecting, from a set of candidate prediction operations each definingat least a prediction direction, a prediction operation for predictionof samples of a current region of a current image, the current regioncomprising an array of two or more rows and two or more columns ofsamples;

deriving intra-image predicted samples of the current region withrespect to one or more of a group of reference samples of the same imagein dependence upon a prediction direction, defined by the selectedprediction operation, between a current sample to be predicted and areference position amongst the reference samples;

in which, for at least some of the candidate prediction operations, thegroup of reference samples comprises two or more parallel linear arraysof reference samples disposed at different respective separations fromthe current region;

detecting whether samples corresponding to any of the two or moreparallel linear arrays of reference samples are unavailable for use inprediction of samples of the current region and inhibiting selection bythe selecting step, if any of the two or more parallel linear arrays ofreference samples are unavailable, of a candidate prediction operationdependent upon the unavailable reference samples.

23. Computer software which, when executed by a computer, causes thecomputer to carry out a method according to clause 22.24. A machine-readable non-transitory storage medium which storessoftware according to clause 23.

1-24. (canceled)
 25. An image encoding apparatus comprising circuitryconfigured to: select, from a set of candidate prediction operationseach defining at least a prediction direction, a prediction operationfor prediction of samples of a current region of a current image, thecurrent region comprising an array of two or more rows and two or morecolumns of samples; derive predicted samples of the current region withrespect to one or more of a group of reference samples of the same imagein dependence upon a prediction direction, defined by the selectedprediction operation, between a current sample to be predicted and areference position amongst the reference samples; in which, for at leastsome of the candidate prediction operations, the group of referencesamples comprises two or more parallel linear arrays of referencesamples disposed at different respective separations from the currentregion; and detect whether samples corresponding to any of the two ormore parallel linear arrays of reference samples are unavailable for usein prediction of samples of the current region and, if any of the two ormore parallel linear arrays of reference samples are unavailable, toinhibit selection, by the selector, of a candidate prediction operationdependent upon the unavailable reference samples.
 26. The apparatusaccording to claim 25, in which the circuitry is further configured todetect for a current region of a current image which, if any, of the twoor more parallel linear arrays are available for use in prediction ofsamples of the current region and to allow selection as a predictionoperation, by the circuitry, of a candidate prediction operationdependent upon at least one of the available parallel linear arrays. 27.The apparatus according to claim 25, in which: for at least some of thecandidate prediction operations, the group of reference samplescomprises one or more predetermined values; the circuitry is furtherconfigured to detect for a current region of a current image if none ofthe two or more parallel linear arrays is available for use inprediction of samples of the current region and to allow selection as aprediction operation, by the circuitry, of a candidate predictionoperation for that current region dependent upon the predeterminedvalues as reference samples.
 28. The apparatus according to claim 25, inwhich the circuitry is responsive to an image location of the currentregion.
 29. The apparatus according to claim 28, in which the circuitryis configured to detect whether the current region is adjacent an edgeof the current region such that sample locations corresponding to atleast some of the two or more parallel linear arrays of referencesamples lie outside the current image.
 30. The apparatus according toclaim 28, in which: the image encoding apparatus is configured to encodethe current image as successive coding units of samples; the imageencoding apparatus comprises a sample store to store samplescorresponding to a region of the current image other than the currentregion; and the circuitry is configured to detect whether, for an imageregion at a location within a coding unit adjacent to a previous codingunit, any of the two or more parallel linear arrays of reference samplesare held by the sample store.
 31. The apparatus according to claim 25,in which the circuitry is configured to detect, separately for referencesample positions disposed vertically or horizontally with respect to thecurrent image region, whether samples corresponding to any of the two ormore parallel linear arrays of reference samples are unavailable for usein prediction of samples of the current region and, if any of the two ormore parallel linear arrays of reference samples are unavailable forreference sample positions disposed vertically or horizontally withrespect to the current image region, to inhibit selection, by theselector, of a candidate prediction operation dependent upon theunavailable reference samples.
 32. The apparatus according to claim 25,in which the circuitry is responsive to configuration data defining oneor more parameters of a decoding apparatus to detect whether samplescorresponding to any of the two or more parallel linear arrays ofreference samples will be available for use in prediction of samples ofthe current region at the decoding apparatus.
 33. A video storage,capture, transmission or reception apparatus comprising the apparatusaccording to claim
 25. 34. An image decoding apparatus comprisingcircuitry configured to: select, from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples; derive predicted samples of the currentregion with respect to one or more of a group of reference samples ofthe same image in dependence upon a prediction direction, defined by theselected prediction operation, between a current sample to be predictedand a reference position amongst the reference samples; in which, for atleast some of the candidate prediction operations, the group ofreference samples comprises two or more parallel linear arrays ofreference samples disposed at different respective separations from thecurrent region; and detect whether samples corresponding to any of thetwo or more parallel linear arrays of reference samples are unavailablefor use in prediction of samples of the current region and, if any ofthe two or more parallel linear arrays of reference samples areunavailable, to inhibit selection, by the circuitry, of a candidateprediction operation dependent upon the unavailable reference samples.35. The apparatus according to claim 34, in which the circuitry isfurther configured to detect for a current region of a current imagewhich, if any, of the two or more parallel linear arrays are unavailablefor use in prediction of samples of the current region and to allowselection as a prediction operation, by the circuitry, of a candidateprediction operation dependent upon at least one of the availableparallel linear arrays.
 36. The apparatus according to claim 34, inwhich: for at least some of the candidate prediction operations, thegroup of reference samples comprises one or more predetermined values;the circuitry is further configured to detect for a current region of acurrent image if none of the two or more parallel linear arrays isavailable for use in prediction of samples of the current region and toallow selection as a prediction operation, by the circuitry, of acandidate prediction operation for that current region dependent uponthe predetermined values as reference samples.
 37. The apparatusaccording to claim 34, in which the circuitry is responsive to an imagelocation of the current region.
 38. The apparatus according to claim 37,in which the circuitry is configured to detect whether the currentregion is adjacent an edge of the current region such that samplelocations corresponding to at least some of the two or more parallellinear arrays of reference samples lie outside the current image. 39.The apparatus according to claim 37, in which: the image decodingapparatus is configured to decode the current image as successive codingunits of samples; the image decoding apparatus comprises a sample storeto store samples corresponding to a region of the current image otherthan the current region; and the circuitry is configured to detectwhether, for an image region at a location within a coding unit adjacentto a previous coding unit, any of the two or more parallel linear arraysof reference samples are held by the sample store.
 40. The apparatusaccording to claim 34, in which the circuitry is configured to detect,separately for reference sample positions disposed vertically orhorizontally with respect to the current image region, whether samplescorresponding to any of the two or more parallel linear arrays ofreference samples are unavailable for use in prediction of samples ofthe current region and, if any of the two or more parallel linear arraysof reference samples are unavailable for reference sample positionsdisposed vertically or horizontally with respect to the current imageregion, to inhibit selection, by the circuitry, of a candidateprediction operation dependent upon the unavailable reference samples.41. A video storage, capture, transmission or reception apparatuscomprising the apparatus according to claim
 34. 42. An image encodingmethod comprising: selecting, from a set of candidate predictionoperations each defining at least a prediction direction, a predictionoperation for prediction of samples of a current region of a currentimage, the current region comprising an array of two or more rows andtwo or more columns of samples; deriving intra-image predicted samplesof the current region with respect to one or more of a group ofreference samples of the same image in dependence upon a predictiondirection, defined by the selected prediction operation, between acurrent sample to be predicted and a reference position amongst thereference samples; in which, for at least some of the candidateprediction operations, the group of reference samples comprises two ormore parallel linear arrays of reference samples disposed at differentrespective separations from the current region; detecting whethersamples corresponding to any of the two or more parallel linear arraysof reference samples are unavailable for use in prediction of samples ofthe current region and inhibiting selection by the selecting step, ifany of the two or more parallel linear arrays of reference samples areunavailable, of a candidate prediction operation dependent upon theunavailable reference samples.
 43. A non-transitory computer readablemedium including computer program instructions, which when executed by acomputer, causes the computer to perform the method of claim
 42. 44. Animage decoding method comprising: selecting, from a set of candidateprediction operations each defining at least a prediction direction, aprediction operation for prediction of samples of a current region of acurrent image, the current region comprising an array of two or morerows and two or more columns of samples; deriving intra-image predictedsamples of the current region with respect to one or more of a group ofreference samples of the same image in dependence upon a predictiondirection, defined by the selected prediction operation, between acurrent sample to be predicted and a reference position amongst thereference samples; in which, for at least some of the candidateprediction operations, the group of reference samples comprises two ormore parallel linear arrays of reference samples disposed at differentrespective separations from the current region; detecting whethersamples corresponding to any of the two or more parallel linear arraysof reference samples are unavailable for use in prediction of samples ofthe current region and inhibiting selection by the selecting step, ifany of the two or more parallel linear arrays of reference samples areunavailable, of a candidate prediction operation dependent upon theunavailable reference samples.
 45. A non-transitory computer readablemedium including computer program instructions, which when executed by acomputer, causes the computer to perform the method of claim 44.